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Abstract: We consider small factor analysis models with one or two fac- 
tors. Fixing the number of factors, we prove a finiteness result about the 
covariance matrix parameter space when the size of the covariance matrix 
increases. According to this result, there exists a distinguished matrix size 
starting at which one can determine whether a given covariance matrix 
belongs to the parameter space by determining whether all principal sub- 
matrices of the distinguished size belong to the corresponding parameter 
space. We show that the distinguished matrix size is equal to four in the 
one-factor model and six with two factors. 
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1. Introduction 

Suppose we observe a sample of multivariate normal random vectors and wish to 
test whether their covariance matrix is diagonal. The likelihood ratio test for this 
problem involves the determinant of the sample correlation matrix. Therefore, 
this test cannot be used if the sample size n is smaller than the number p 
of entries in the random vectors, because the sample correlation matrix will 
always be singular. A nice way around this problem was proposed in Schott 
(2()0.")), where the sum of squared pairwise sample correlations is used as a test 
statistics. The relevant distribution theory involves a central limit theorem in 
the paradigm where n,p — > cx) such that p/n — > c e (0, oo). 

Why is it possible to prove such a limit theorem and which situations are can- 
didates for development of similar results and associated statistical techniques? 
We believe that a fundamental aspect of these questions is a property of finite- 
ness. In the above problem this property amounts to the ability to determine 
whether a covariance matrix is diagonal by verifying whether each principal 
2x2 submatrix is diagonal. The squared sample correlations summed up in the 
test statistics of Schott (2005) do exactly that. In this paper we show that such 
finiteness structure arises more generally in factor analysis models. 

The factor analysis model for p observed variables and with m factors is the 
set of multivariate normal distributions Afp{fJ., S) with arbitrary mean vector 
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and a covariancc matrix E in the set 

Fp^rn = {A + rr* : A positive definite and diagonal, T e MP"""} . 

For background on this classical statistical model see, for instance, Anderson 
and Rubin (1956); Harnian (1976). Note also that for m = it is reasonable 
to define _Fp.o to be the set of positive definite and diagonal p x p matrices. In 
this paper we establish the following result that resolves part of a conjecture in 
Drton et al. (2007); see also open problem 7.8 in Drton et al. (2009). 

Theorem 1. Suppose m G {0,1,2}, and let S = (cy ) be a positive definite 
matrix of size p x p with p > 2(m + 1). Then S is in Fp,m if o,nd only if 
the principal submatrix Y,a.a = {<^ij)i.jeA is in F2[rn+i),m for each index set 
A C {1, . . . ,p} of size 2(m + 1). 

As motivated above, this result is of statistical interest as it suggests that 
in a high-dimensional setting with large number of variables p a test of the 
factor analysis models can be carried out by testing smaller marginal hypotheses 
concerning only 2(m + 1) variables. While our result provides the theoretical 
basis for such tests, an appropriate distribution theory still has to be worked 
out. This, however, is a topic beyond the scope of this note. 

The remainder of the paper is devoted to the proof of Theorem 1. In Section 2, 
we outline our approach to the finiteness problem and resolve the one-factor case. 
In Section 3, we tackle the more complicated case of two factors. Concluding 
remarks are given in Section 4. 

2. Approach to the problem and one- factor models 

We first introduce some notational conventions. Suppose / and J are two index 
sets. In this paper / is typically of the form [p] = {1, 2, . . . ,p} or {2, 3, . . . ,p} 
with p S N. For any finite set A, let \A\ denote its cardinality. Suppose A = 
{Kj)iei.jeJ is an \I\ x \J\ matrix. For A C I and B C J, we let Aa.b = 
i^ij)ieA.jeB denote the |^| x \B\ submatrix. When using the complement of a 
set A as an index set we write \A as in 

^\A,\B ■= ^I\A,J\B- 

As a further shorthand, we let \i := \{i}, Aa Aa^j and A_b Aj b- 

If E G Fp^rn, then the representation S = A + A A* is not unique because we 
may multiply an orthogonal matrix to A from the right. However, the diagonal 
matrix A, and thus, the positive semi-definite and rank m matrix A A* may be 
unique. We will repeatedly use the following lemma to establish such uniqueness. 

Lemma 2. Let p > 2m + 1, and consider two p x p matrices 5* = (ipij) and 
4> ~ {4>ij) with the same off-diagonal entries and same rank m. For any i G [p], 
if we can find two disjoint subsets A, B C. [p] \ {i} of cardinality \A\ = \B\ = m 
such that dct{'^ a.b) 0, then ipu — (\)n. 
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Proof. Since 5* and <!> arc both of rank m, the foUowing two (m + 1) x (to + 1) 
minors are equal to zero: 



0. 



The above two minors are entry-wise the same except for ipn and 
det{^ a,b) = det(<I>yi^s) 7^ 0, it follows that = (pa. 



i. Since 
□ 



The next lemma will provide a way to give an induction-based proof of The- 
orem 1. 

Lemma 3. Let p > 2m + 3, and suppose that E G W^p is a positive definite 
matrix that has all (jj — 1) x [p — 1) principal submatrices in -Fp-i^m- Write 



id = D + GG* 



with A and D positive definite and diagonal, and F, G G M^J'^^)^'" . To see the 
correspondence with the original matrix S clearly, label the rows of D and G by 
2,3, ... ,p. Then the following two conditions imply that E belongs to Fp^„i-' 



(i) The two matrices V and G satisfy T\^i = G 



(a) There are disjoint subsets B,C C ([p] \ {l,p}) of cardinality \B\ ~ \C\ 
TO such that detCEs.c) 7^ 0. 

Proof. From A = F = (7^ ), D and G form the matrices 



A: 



Sii 
D 



and F 



which is a matrix in Fp^m- We claim that S 



2 rows and it holds that cr.y = cr.y 



Let S = A -f FF* 

By assumption (i), F and G agree in p 
except possibly for the pair (z, j) = In order to show that dip = aip, we 

use the index sets B and C from conditon (ii) to construct the following three 
(m -I- 1) X (to 4- 1) minors, which are all equal to zero: 



yip 



CTlp 



flp 



0. 



(1) 



The first minor is zero because {1} U S and G U {p} are two disjoint subsets of 



cardinality m -I- 1 and because S is in Fp.m. Recall that rank(FF*) 



The 



last minor is zero for the same reason. Recall that we assume that p > 2m + 3, 
whereas the first principal submatrix shown in (1) involves only 2to-|-2 different 
variables. The middle minor is zero because it is entry-wise equal to the first 
minor. Since by assumption det{T,B.c) 7^ 0, Lemma 2 yields that aip = aip. □ 

We are now ready to study the case of to = 1 factor. 

Theorem 4 (to = 1). A positive definite matrix E G W^p with p > A belongs 
to Fp^i \ Fp^Q if and only if every 4x4 principal submatrix belongs to F^^i and 
at least one 2x2 principal submatrix does not belong to i^2,o- 
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Proof. The 'only if part is immediate. We prove the 'if part by induction. The 
induction base is p = 4 in which case the statement is vacuous. Now suppose 
that p > 5 and that S has every principal 4x4 submatrix in i^4_i and at least 
one non-zero off-diagonal entry. If no other off-diagonal entry is non-zero, then 
E = A + rr* for a vector F £ Rp with exactly two non-zero entries. So, we may 
assume that E has at least two non-zeros entries. Without loss of generality, 
suppose that (J23 7^ and a^i ^ 0. 

By the induction hypothesis, Y,\p \p and Eyj^^^j^ belong to Thus we are 

able to write 

S\p,\p = A + rr* and E\i,\i =D + GG* 
with r, G G M^^^. Therefore we have 2 representations of E\{i pj y{^i_p}, namely, 

S\{i,p},\{i,p} = A\i,\i -t- T\iT\^ = D\p^\p + G\pG\p. 

Note that again we assign row indices to the matrices D and G based on the 
correspondence to the original matrix E, so we use I?\p \p and G\p \p instead 
of £'\(p-i),\(p-i) and G\(p_i)_\(p_i) respectively. Since (723 = 7273 7^ and 
tT34 = 7374 ^ 0, we deduce that (T24 = 7274 0. Using these three non-zero 
entries of E and applying Lemma 2, we know that F^iF* = G^pG*^. Therefore, 
Fyi and G\p can only differ by a sign. Changing the sign of G if necessary, we 
obtain that Fyj^ = G\p and thus Lemma 3 implies that the induction step goes 
through. □ 

3. Two- factor models 

If F e IRP^™ has rank m and G is another matrix in Rp^'" that satisfies GG* = 
FF*. then F = GQ for some orthogonal matrix Q (Anderson and Rubin, 1956, 
Lemma 5.1). However, this holds more generally. 

Lemma 5. If T,G G Qj-g matrices with dimensions p > m that satisfy 

GG* ~ FF*, then F — GQ for some orthogonal matrix Q. 

Proof. Suppose F has rank k < m. Then rank(G) ~ k also. There are now 
orthogonal transformations Qi and Q2 such that FQi = (F',0) and GQ2 = 
(G',0), where F' and G' are full rank p x k matrices. According to the above 
fact, F' = G'Q for some orthogonal k x k matrix Q. Let = diag{Q, Im-k)- 
Then Q2Q3Q1 is an orthogonal matrix and F = GQ2Q3Q1. ^ 

By Lemma 5, if the matrix E^^^i p} \{i pj has a unique representation as the 
sum of a positive definite and diagonal matrix plus a positive semi-definite 
matrix of rank m, then conditon (i) in Lemma 2 can be satisfied by applying an 
orthogonal transformation. We thus need to study when this representation is 
unique and how uniqueness may fail for m = 2 factors. We begin our discussion 
by considering a 5 x 5 matrix. This prepares us for an induction step from p = 6 
to p = 7 because in this step 5]\{i,p},\{i,p} '^^ ^^^^ 5x5. 
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Lemma 6. Suppose S is a 5 x 5 positive definite matrix that can he written as 

E^A + rr*, (2) 

where A is positive definite and diagonal, and T G MP'^'^ has rank 2. Let k be 
the largest integer n for which there is an index set A C [5] of cardinality n 
with rank(r/i) = 1. Then under the assumptions that S has no representation 
like (2) with rank(r) < 1, and every row of Yi contains at least one non-zero 
ofj-diagonal entry, it holds that k < 3 and we have the following: 

(i) If k = 1 or k = 2, then the representation in (2) is unique, that is, 

rr* = GG* for any other representation Y, = D + GG* . 
(a) Ifk = 3, then after a permutation of rows we may assume i/iai rank(r[3]) = 
1 . Then there exists an orthogonal matrix Q such that 



(ill 

721 
731 
741 
V751 






742 
7^2/ 



where primes indicate entries that are necessarily non-zero. Moreover, for 
any other representation S = D + GG*, it holds that G ~ {gij), when 
brought into the same form as T by an orthogonal transformation, shares 
the first column with T and satisfies g'429'52 ~ 742752- 



Proof. Since every row of S is assumed to contain a non-zero entry, no row of T 
is zero. Assuming that rank(r) < 1 is not possible in a representation of S, we 
must have that fc < 4. If fc = 4, by applying an orthogonal transform, we can 
write r as 



r = 



751 





752 



But then wc have a contradiction to our assumptions because we can represent 
E as 



A [4] 









2 

-7g2 



^ [4],1 
751 



^ [4],1 
751 



Hence, we must have A; < 3. 

Now suppose there is another representation S = Z) + GG* . Let 4' = FF* and 
$ = GG*. There are two cases. 



(a) k ~ 1,2: If k = 2, we may assume rank(F[2] 
onal transform, we can write F as 



1. By applying an orthog- 



(ill 


M 


721 





731 


732 


741 


i2 


\751 


ij 
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(b) 



where we use again primes to highlight non-zero entries. Since fc < 3, the 
submatrix 4 5} has rank 2, and thus we have enough non-zero 2x2 
ofF-diagonal minors of $ to apply Lemma 2 and deduce uniqueness of the 
representation. If fc = 1, then Lemma 2 applies immediately. 
fc ~ 3: Let us assume rank(r[3]) = 1, in which case we can write F as 



(ill 




ill 





731 





741 


742 


\751 


7^2/ 



Since the off-diagonal minor 



?/'34 
V'52 ?/'54 



Lemma 2 yields that 'i\)ii = 0ii. Similarly, we obtain that "022 = 022 and 
"033 = '/'33- It follows that rank(G[3]) = 1, and thus we can write G as 



G 



(ill 

ill 
ii 

.941 
V.951 






542 

552/ 



Since ■0i4 
know : 



= 7n74i = 014 = 7nff4i and V^is 
541 and 751 = 1751. Therefore, 



7n75i = 015 = 7n55i, we 



G = 



/7;i 

721 

7^1 

741 

V751 






542 

552/ 



For this case the representation is not unique, but since "045 
hold that 742752 



^145 



542552- 



it must 



□ 



Equipped with Lemma 6, we are able to prove finiteness for to = 2 factors. 

Theorem 7 (to = 2). A positive definite matrix S G W^'^ with p > 6 belongs 
to Fp^2 \ Fp^i if and only if every principal 6x6 submatrix belongs to Fq 2 and 
at least one 4x4 principal submatrix does not belong to F^^i. 

Proof. As in the proof of Theorem 4, only the induction step requires work. So 
suppose that every principal 6x6 submatrix of S is in i^6,2- By the induction 
hypothesis, all the (p — 1) x (p — 1) principal submatriccs belong to -Fp-1,2; and 
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without loss of generality, we may assume that S|2, 
submatrix not in i^4.i. 

Suppose that some row of E 



,6}, {2,. ..,6} 



contains a 4 x 4 



{2,....6},{2,...,6}j say the first one, has all off- 
diagonal entries equal to zero. Consider the representation S\p,\p = A + FF*. 

Then rank(F{3 4 5 e}x[2]) = 2, for otherwise S|2 6}. {2,. ...6} would be in ^5,1- It 

follows that r2 = (0, 0) and all the off-diagonal entries in the second row of E 
except for the last one are zero. By considering a representation of the matrix 
Eyi \i instead, we can deduce that in fact all of the off-diagonal entries in the 
second row of S are zero. Hence, the induction step goes through easily as we 
can insert a row of zeros into the matrix F in a representation E\2,\2 A-|~FF*. 

In the remaining cases, we can assume that the matrix Ej2....,6}.{2,...,6} sat- 
isfies the two conditions in Lemma 6. If E|2,...,6},{2,...,6} belongs to case (i) of 
Lemma 6, then the center matrix E\{i \|i pj. has a unique representation. To 
see this, note that by Lemma 6, E{2,...,6},{2,...,6} has a unique representation, and 
that we can use a non-zero off-diagonal 2x2 minor from E|2,...,6},{2 6} to de- 
duce the uniqueness of the representation of E\{i pj \{i pj. Therefore, Lemma 3 
implies that the induction step goes through in this case. 



Now assume that E 



write E 



\p,\p 



A + FF* and E 



{2,...,6},{2,...,6} belongs to case (ii) of Lemma 6. We first 
■^^ .^^ = D + GG\ where F and G are (p - 1) x 2 



matrices. By Lemma 6, F and G have the following typical forms 



/ 7ii 


712 \ 


721 





731 





7^1 





751 


752 


761 


762 


771 


772 


7p-i4 


7p-l,2 



v 



/ 



and G — 



J 



\ 



721 
731 

7^1 

751 
761 
971 







.952 
.962 
572 



ffp-ia 5p-l,2 

\ 5p,i 5p,2 / 



where we assigned row indices based on the correspondence to the rows in E. 

If at least one of the entries 772, . . . , 7p-i,2 or 372, . . . , 5p-i,2 is non-zero, then 
Lemma 2 implies uniqueness of the representation of E\{^i pj pj, which allows 
to apply Lemma 3. Otherwise, we only know that 752752 = 552562- K (752; 752) = 
(5521562) or 712 = or (7p_2 = 0, then the induction step goes through. So we 
are left with the case, where 772 = ■ ■ ■ ~ 7p-i,2 = 572 = ■ • • = .9p-i,2 ~ 0, and 
712 ^ 0, gp.2 ^ 0. Multiplying the second column of G by —1 if necessary, we 
can assume that 752 and have the same sign. To complete the proof we will 
show that in this case 752 = 552- 

Consider the submatrix S = E|]^ 3 4 5 g p} |i 3 4 5 g pj.. From the representation 
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of S 



\p,\p 



we obtain that 



5' 



\6A6 



^{1,3, 4, 5, 6}, {1,3, 4, 5, 6} + 



Similarly, the representation 



'S'\l,\l — -D{3,4,5,6,p},{3,4,5,6,p} + 



/711 

7^1 

751 
V761 



/7^i 

7^1 

751 
761 



il2\ 




752 
762/ 



552 
562 



/71I 

7^1 
7^1 

751 
V761 



/7^i 
7^1 

751 
761 



7;2\ 




752 
7^2/ 



\ 


.952 
.962 



\9pi 5p2/ \9pi 5p2/ 



(3) 



(4) 



is inherited from the representation of S]\i_yi. Since S € ^6,2, it also has a 
representation S = C + FF^ as in (2) with C = (cy ) diagonal. We label the 
rows and columns of C by {1, 3, 4, 5, 6,p} based on the correspondence to the 
rows in E. Using the structure of the two representations of S\i \i and 5^6, \6 in 
(3) and (4), we can deduce via Lemma 2 that ^55 = C55 = ^55; again we assign 
indices for ^55 and ^55 according to the correspondence to rows in S. It follows 



that 7I1 - 



-7I2 



751 +ff52- Having assumed that 752 and 552 have the same sign, 



we conclude that 752 — (752- 



□ 



4. Conclusion 

Our main result. Theorem 1, shows that for m < 2 factors the covariance ma- 
trices of distributions in the factor analysis model possess a finiteness structure. 
This also of interest for recent work on the algebraic geometry of the two-factor 
model (SuUivant, 2009). Our proof uses only linear algebra and shows that, in 
the covered cases, the distinguished matrix size for finiteness is 2(to -|- 1), that 
is, one can decide whether a covariance matrix belongs to Fp^^n by only looking 
at 2(m + 1) X 2(m + 1) principal submatrices. Unfortunately, our arguments 
seem difficult to extend to the cases with m > 3 as one would need to show that 
larger off-diagonal minors do not vanish in certain situations. 

In unpublished work concerning a closure of the model, Draisma (2008) shows 
that finiteness holds also for an arbitrary number of factors to. However, his 
method of proof does not provide the distinguished matrix size at which finite- 
ness occurs. It is natural to conjecture that this matrix size is equal to 2(m + 1) 
in general. The following example clarifies that the distinguished matrix size 
cannot be smaller. 

Example 8. Consider the matrix 

_ / 2/„j_|_i Irn+l \ 
\ Im+l 2/m+l J 
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where Im+i is an {rn + 1) x (to + 1) identity matrix. Then every (2m + 1) x 
(2m + 1) principal submatrix belongs to -Pjm+i.m- For example, the following 
matrix obtained by deleting one row and one column of S can be written as 





Nevertheless, E ^ F2m+2,m, because the off-diagonal block S[m+i].[2m+2]\[m+i] 
has rank (m + 1). 
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